Measuring Knowledge Acquisition in Dynamic Deci- sion Making Tasks

When evaluating the effectiveness of interactive learning environments it is important to include measures of knowledge acquisition that complement measures of performance. In this paper we report on participants’ knowledge acquisition in a dynamic decision making task where participants learned about and managed a small developing nation. In the course of the experiment participants not only had to make decisions but also answer multiple-choice questions and short essay questions. The results suggest that participants had a fairly good understanding of the reinforcing nature of national development processes and of processes that are in close causal proximity to their decisions. On the other hand, participants largely failed to recognize nonlinearities, the existence of the outflows to stocks and the proper treatment of delays with different durations. Knowledge acquisition was facilitated by the intensity of participants’ exploration activities during a simulation-based, guided exploration phase between reading textual instructions and making actual, simulation-based decisions.